Comparing Player Skill, Game Variants, and Learning Rates Using Survival Analysis

Game designers can use computer-aided game design methods to quantitatively compare player skill levels, different game variants, and learning rates, for the purpose of modeling how players will likely experience a game. We use Monte-Carlo simulation, hazard functions, and survival analysis to show how difficulty will quantitatively change throughout a game level as we vary skill, game parameters, and learning rates. We give a mathematical overview of survival analysis, present empirical data analyses of our player models for each game variant, and provide theoretical probability distributions for each game. This analysis shows the quantitative reasons why balancing a game for a wide range of player skill can be difficult; our player modeling provides tools for tuning this game balance. We also analyze the score distribution of over 175 million play sessions of a popular online Flappy Bird variant to demonstrate how learning effects can impact scores, implying that learning is crucial aspect of player modeling.